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A Study of the Feasibility of Statistical Analysis 
of Airport Performance Simulation 

!• Introduction 

The main objectives of studies planned by FAA/NASA implicitly re- 
quire an adequate definition of airport capacity with view toward sorting 
out the roles of pilot, ground controller, and other factors that might 
Influence the overall safety of our aviation system. Any future proposed 
simulation studies by NASA and FAA should, of course, be carried out with 
the aid of sound statistical tools. The study described here is a funda- 
mental one in which the feasibility of conducting statistical analysis 
of such simulation experiments is investigated. 

At the outset of the study, we attempt to study the form of the 
distribution of airport capacity in the standard simulation model (Ref 4 
(2)). This capacity measure would be the basic response in any planned 
simulation experiment designed to study the roles of such variables as 
number of pilots, number of controllers, number of planes, environmental 
conditions, etc. on airport performance. 

After the distribution of throughput capacity is assessed, an attempt 
is made to determine how effective standard experimental design and analysis 
of variance techniques would be in detecting capacity changes when conditions 
vary. In particular, the crucial problem here is to first determine the 
effect of the non-Gaussian Distribution of capacity on standard analysis 
of variance techniques and power calculations. Adjustments must be made 
in power calculations to account for the natural heterogeneity of variance 
in capacity. Secondly, power computations must be made in order to 



determine how economic simulation experiments would be if they are designed 
to detect capacity changes from condition to condition. In the latter, 
we are preoccupied with determining the number of replications in the sim- 
ulation experiments that will result in an effective study, i.e., one 
that is able to detect small or moderate changes in mean capacity. 

Many of the conclusions drawn here are a result of statistical sim- 
ulation studies, i.e., studies in which significance level and power of 
standard tests are empirically studied on the computer via Monte-Carlo 
Techniques, and non standard conditions (non Gaussian and hetereogeneous 
variance) are built into the simulation, 


II. Capacity Measure 


The capacity measure to be used is given by the airport throughput 


rate 


C = no. of vehlcles/hour , 


( 2 . 1 ) 


Hereafter the measure C will denote the mean throughput , a population 
parameter often called capacity. The inter-arrival time, lAT, repre- 
sents the time between touchdowns. Because of the procedure inherent 
to the simulation that "buffers" the touchdowns, one can realistically 
assume that lAT follows a normal distribution with parameters (y,o). 

In addition, the buffer provides that y and a be related in the simula- 
tion through the following relationship (ref. 2) 


3600s 


lAT 


^B*^IAT 
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where 


s * separation In nautical miles 
V = approach velocity, knots 
Z„ = buffer factor (Gaussian probability factor) 

If s • 2, V » 140 and Z„ “ 1.96, the following approximate relationships 

B 

hold: 

“lAT ' 

3600 

V.* “ '' • 

‘^lAT 

Is measured In seconds. The capacities of Interest In this study 
range from 20 to 60 vehicles/hour. 

(a) Distribution of Capacity 


It is of Interest to Initially consider the distribution of through- 
put rate, which of course Is a random variable. Let lAT be a random 
variable denoted by y. Strictly speaking, y has a practical lower 
bound and thus follows a truncated normal distribution . For now, assume 
the point of truncation to be 6; thus the density of y Is known to be 


f(y) “ K ^e y>6 

where 


(2.3) 


K 





dy 


to begin, we need the distribution of the throughput z ■= 1/y. Following 
standard procedures 
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(2.4) 


-l/2o2(i-y)2 

f(z) = K”^e 



-l/2o2(^y)2 

= K-le 




One can easily see that the above density, that of the Inverse of a 
truncated normal, integrates to unity. For, we can write 


rl/8 

f(z)dz = K ^ 

^0 


rl/6 
• 0 


-1/20^ (-^-y) 2 . 
e ^ 4- dz 


(2.5) 


and Invoke the transformation — = w, allowing dz = -z^dw. Thus (2.5) 

z 

is written 


5 . 

K“^ e w^(-^)dw 


5 

But e dw = -K and thus (2.5) equals unity. 

j oo 

Figures I - VI give an indication of the appearance of the distri- 
bution of z. Note of course that in many cases the throughput is far 
from Gaussian. 

It is of interest to work out the moments of the distribution of 
z and also attempt to gain some Insight on the distribution shape. An 
approximation for the variance of the distribution can be helpful in 
what follows. For the first moment 


E(z) = 


■ 1/6 ^ - 1 / 202 (^ y )2 

— e 

0 


k 


z 


dz • 



Allowing 


1 

z 


V, we have 


E(z) 


K‘l 





= K"l 


P jL ^-l/2a^Cv-y)2 

h ^ 


similarly, one can show that 

E(z 2) - K-» r 

The above suggest that in certain situations, a simple approximation of 

Var(z) ■ might be found in terms of the parameters of inter-arrival 
z 

time, namely p and o. A Taylor Series approximation approach should be 
quite good for cases where p»o. For example, expanding 1/y^ and 1/y 
in a Taylor Series around y = p, 
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This type of approximation of inverse moments is not uncommon for cases 
where y > 3o (See Kendall & Stuart [1]). In our situation, our mean 
capacity range of 20 - 60 results in the following approximate ranges 
on y and a. 


C (veh/hr) 
60 
51 
45 
40 
36 
30 

25.7 

20 


60 

70 

80 

90 

100 

120 

140 

180 


5 

10 

15 

20 

25 

35 

45 

65 


It Is clear that this approximation for Var(z) should be quite good 
until we reach the very low end of the capacity specifically when capac- 
ity is near 20 veh./hr. The approximation itself can be "checked" by 
simulation results. The purpose of the approximation lies in its use 
in developing an algorithm for computing power. This will be discussed 
in a later section. 

III. Review of Analysis of Variance Principles 

As we mentioned earlier, it is necessary to investigate a wide 
variety of conditions on mean throughput to determine if indeed one can 
effectively and economically compare capacity statistically from con- 
dition to condition with simulation experiments. The analysis of 
variance procedure results in obvious problems. These problems will 
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be iqore obvious after we review analysis of variance procedures ^^nd the 
calculation of power of the analysis. 

(a) Model and Assumptions 


Briefly, when k group means are to be compared in an analysis of 
variance, a model Is assumed of the type 


'ij 


'<1 * '13 


1=1,2, , . . ,k 
j=l , 2 , . . . ,n . 


(3.1) 


where y. Is the group population mean, e.. Is the disturbance or random 
1 ij 

error term of the model. The is the basic response measurement 

under the observation In the l*”^ group. For our case, of course, 

til til 

z^j would represent the j simulated throughput in the 1 group and 
the mean throughput or capacity for the 1*"^. One Is Interested In 
testing then 



= Y2 = ... = 

There Is a difference In the means 


The analysis of variance procedure then (see Walpole and Myers [2]) 
provides a methodology for testing this hypothesis. The usual test 
statistic Is an F-statlstlc. One can consult [2] for the details. 
Actually, In our application It Is quite likely that the model In 
equation (3.1) could and should be extended to cover the study of 
several groups simultaneously. For example In a so-called two way 
classification, one might want to simultaneously study number of pilots 
and number of ground controllers (and perhaps interaction) as far as 


7 


their effect on capacity is concerned. 

Regardless of the number of factors involved, certain strict assump- 
tions must be considered in analysis of variance problems. They need to 
be discussed here, simply because they fall to hold in the setting of our 
problem. In the model of equation (3.1) it is assumed that the are 
Gaussian with mean zero and common variance These assumptions are 

necessary in order that the F-test used in the procedure be valid. 

(b) Power of the Analysis of Variance 

The analysis of variance procedure discussed here actually entails 
a test of the hypothesis that the means of each group, or the so-called 
"treatment means" do not differ from one another. Since the methodology 
falls into this traditional framework, the decision regarding the number 
of simulation runs, i.e., replications, the number of treatments or 
treatment combinations (number of conditions in our case) should be 
made with view toward obtaining a certain power of the test. The power 
of a test is defined as follows 

P(a,v^,V 2 ,*) = Pr[reject Hq|h^]. 

The power calculation is not a difficult one, in general, and if all of 
the above assumptions hold, involves the use of the non-central F dis- 
tribution [3]. The distribution Involves the following four parameters, 

k = number of treatments 

= numerator degress of freedom = k-1 

\>2 “ denominator degrees of freedom =5 N-k, where N is the total 
number of observations 
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(.3,2) 


i 


ka^ 


The quantity 9 is called the non-centrallty parameter . The power of the 
test increases monotonically with an increase in V 2 » and of course 
the parameter $. As one can easily see, the test becomes more 
powerful or more sensitive to mean differences as the treatment means 
become more dispersed and as becomes smaller, and as the number of 
replications becomes larger. 

The most popular mode for finding the power of analysis of variance 
is the use of Operating Characteristic Curves or 0,C. curves. Examples 
are displayed in Figures VII - X. The ordinate on the plots is actually 
the probability of making a Type II error, or the probability of false 
acceptance of Hq. Thus the power is found by calculating the complement 
of the number found on the abscissa. For example, let us assume that 
all standard conditions are met for analysis of variance and that there 
are 4 treatments to compare with 5 observations or replications per 
treatment. In addition, let us assume that the treatment means are 
= 55, Y 2 “ Y 3 = 61, and y^ “ 64, The parameter ♦ is given by 
(asstane * 25) 


/5(A6) 


- 1,7 


If one uses a significance level of the test of a = 0,05, the power from 
the chart for = 3 and - 16 is found to be 1 - 0.3 = 0.7. The 
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implication is that if the real world situation provides a set of means 
given by the above, there will be 0,7 probability of rejecting or 
detecting a significant difference in the means. * 

In our application the data to be analyzed would be simulation 
data, and the y^ would be mean throughput or capacity. In the follow- 
ing section we deal with the problem of failure of the analysis of 
variance assumptions. 

IV. Failure of Analysis of Variance Assumptions 

As we Indicated earlier, normality of the , l.e., normality of 
the treatment population is essential in order that the theory associated 
with the analysis of variance be Intact. Also, the homogeneous variance 
assumption must hold. These assumptions allow one to use the F- 
test in analysis of variance. If they do not hold, analysis of variance 
is not correct in the sense that the distribution of the usual test 
statistic does not follow an F -distribution and thus an F would 

Vi,V2 

only be an approximation. That is, if one used an F, his true signifi- 
cance level would theoretically not exactly be what he thinks it is. • 

The same, of course, would be true of the power since the validity of 
the non-central F relies on the same assumptions. The problem then be- 
comes one of assessing how robust (or insensitive) the procedure of 

analysis of variance is to these assumptions. If there is a lack of ' 

robustness, then how can one adjust some feature of the total methodology 
in order to recapture validity. 

I 

f 

J 

I 

i 

5 
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It Is known that analysis of variance is fairly robust to departures 
from the normal distribution assumption. However, there is no known work 
that Investigates this robustness of analysis of variance to the distri- 
bution Involved here, namely the inverse of ^ truncated normal. In the 
case of the homogeneous variance assumption, there is no effect on the 

significance level of the test since under the condition of Hq (equal 

1 1 

means) the variances In — * will be the same for each group (each 
treatment being identically distributed). However, when Hq Is not true, 
there is a lack of homogeneity of variance. Indeed, for IATnN(y^.„,o .».) , 

X/ifX XAJ» 

we found from 11(a) that the approximate variance of throughput (on an 
hourly basis) is given by 


var 


3600 


36002a2 

I AT 


lAT 


where lAT is measured as sec/vehicle. Clearly, then, when one is testing 
the hypothesis that mean throughput (capacity) differs from condition to 
condition, he must cope with the fact that when Hj^ is true, the variances 
also differ. Thus the computation of the power of the test cannot be 
made in the usual fashion because the non centrality parameter in (3.2) 
does not apply; indeed, there is no constant (<^tat playing the 

role of Var ^^^ ) . Thus, some adjustment in the computation of the power 
must be made. 


(a) Simulation Study of Significance Level of Analysis of Variance 

As was indicated earlier, there is a need to determine the adequacy 
of analysis of variance procedures when the basic measurement is through- 
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put. 


3600 


. A simulation study was designed to determine how robust the 


significance level of the usual analysis of variance Is to departures 
from the Gaussian assumption. There Is no need to be conceimed about 
heterogeneous variance here because the significance level a = 

Pr (reject Hq|Hq) would be studied by simulating distributions of 
under the condition that the capacities are equal. 

Various conditions on capacities, number of treatments, number of 
observations per treatment (number of touchdowns at that condition), etc. 
were fixed to use in the computer simulation. The (yi. ^ ^^,g.y^^) combina- 
tions were controlled in order to establish the desired conditions on 

capacity. Basically, the model of equation (3,1) was used to generate 

3600 


and y . In order to provide 

a practical truncation of the normal random variable, the Monte Carlo 


data with each z . . being 

ij y 


Procedure truncated the normal variate at 51, i.e., effectively allow- 
ing no lAT below 51. A computer program was then written to compute the 
F-statistic for 2000 cases for each condition. The distribution of the 
F-statlstlc was then simulated and the probability of exceeding the "true" 
F percentage point was found. This then represents the significance 
level of the tests for analysis of variance in which the basic random 
variable is vehicle throughput. The results are given in Tables I and 
II. As one can easily see, the results strongly suggest that doing 
analysis of variance is quite good even when the basic measurement is 
not Gaussian but rather the inverse of a Guasslan variate. The column 
headed "advertised a" is the significance level that one proceeds with 
in the analysis of variance methodology. The ± values indicate statis- 
tical fluctuation in the simulated significance level. 
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Table I 


Simulated Significance Level 
5 Treatments, 10 Observations per Treatment 


Capacities 

^lAT 

°IAT 

Advertised a 

Actual a 
(Simulation) 

60 

60 

5 

0.05 

0.044810.0024 

51.4286 

70 

10 

0.05 

0.051610.0034 

45 

80 

15 

0.05 

0.049410.00307 

36 

100 

25 

0.05 

0.049610.00208 

30 

120 

30 

0.05 

0.043410.00291 

40 

90 

20 

0.05 

0.052810.00298 

60 

60 

5 

0.10 

0.100510.0055 

51.4286 

70 

10 

0.10 

0.098810.0022 

45 

80 

15 

0.10 

0.099 1 0.0033 

40 

90 

20 

0.10 

0.100210.0045 

30 

120 

30 

0.10 

0.097810.0038 

60 

60 

5 

0.20 

0.199810.0040 

51.4286 

70 

10 

0.20 

0.193 1 0.0041 

45 

80 

15 

0.20 

0.187610.0053 

40 

90 

20 

0.20 

0.198810.0038 

36 

100 

25 

0.20 

0.192 1 0.00322 

30 

120 

30 

0.20 

0.198410.0054 

25.7143 

140 

45 

0.05 

0.048 1 0.0035 
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Table II 


Simulated Significance Level 
4 Treatments, 10 Observations per Treatment 


C 

20 


I AT 


180 


*^IAT 

65 


Advertised oi 


0.05 


Actual a 
(Simulation) 

0.045810.0029 


2 Treatments, 10 Observations per Treatment 


c 

’^lAT 

'^lAT 

Advertised a 

20 

180 

65 

0.05 

20 

180 

65 

0.10 


Actual a 
(Simulation) 

0.043510.0085 

0.096810.0031 
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(b) Simulation Study for Power of Analysis of Variance 


Of course, knowing that the true significance level of the analysis 
of variance is very close to the one applied in the methodology is very 
comforting, but does not necessarily Imply that one can generate a 
reasonable procedure for computing the power of the test; particularly 
since under Hj^, l.e,, when the treatments differ, varying values of y 
imply heterogeneous variances. We would still like to be able to use 
the O.C, charts in Figures VII - X. The question then becomes "What 
is the effective value of in the expression for the non centrality 
parameter In equation (3.2)?" However, when one fixes conditions on 
the capacities with varying y and a, this variance Is not constant. 

As an adhoc methodology to apply In the calculation of consider the 
use of mean ^lAT 


Q 


2 

z 


(3.600)252^T 



f 


thus giving us for $, the relation 3.2 with replaced by the 

"effective" above. Again, simulation was done to study the appro- 
z 

prlateness of this approximation of the power. Table III gives the 
results of this study. 

The results In Table III are promising In that it does Indicate 
a very definite procedure through the use of O.C. charts for approximat- 
ing the power of the analysis of variance test even though the standard 
conditions of analysis of variance are not met , It should be emphasized 
that the above is not exhaustive and computer simulation costs would 
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Table III 


Simulated and Computed Power of Analysis of Variance 


Nominal Capacity = 
4 treatments, 5 observations 

Actual Capacities *^IAT 

h. ^ ^ ^ 

55 58 61 64 65.45 62.06 59.01 56.24 


60, 

per treatment 


2 



51.39 29.58 15.07 6.10 


a 

o2 

z 

$2 

Computed Power 

Simulated Power 


0.05 

25 

2.3 

so. 70 

0.704 ±0.0068 


0.01 

25 

2.3 

SO. 42 

0.436810.0059 



Same as 

Above , 

but 10 observations per 

treatment 


a 

z 

$2 

Computed Power 

Simulated Power 


0.05 

15 

4.6 

so. 98 

0.979610.0055 




Nominal Capacity ^45, 




5 treatments, 5 observations per treatment 


Actual Capacities 


’^lAT 

®IAT 


^1 ^2 ^3 

^4 ^5 

’^i 

»^2 ^"3 ^ ^5 '"l 


“1 

39 42 45 

48 51 

92.3 

85.7 80 75 70.6 435.4 

306.3 212.9 145 

96 

a 

z 

$2 

Computed Power 

Simulated Power 


0.05 

72 

1.25 

so. 36 

0.39310.0057 



Same as 

Above , 

but 10 observations per 

treatment 


a 

o2 

z 

$2 

Computed Power 

Simulated Power 


0.05 

72 

2.5 

SO. 75 

0.750810.0047 



Same as 

Above , 

but 20 observations per 

treatment 


a 

o2 

z 


Computed Power 

Simulated Power 


0.05 

72 

5.0 

so. 99 

0.97810.006 



i, 

.t 

) 


i 
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Table III continued 


Simulated and Computed Power of Analysis of Variance 


Nominal Capacity = 20 
4 treatments, 5 observations per treatment 



25.7 22.5 18 16.4 140 160 200 200 2043.4 3070.1 5748.1 5748.1 


a 

z 

Computed Power 

Simulated Power 

0.05 

53 

so. 28 

0.263610.0049 


2 

Nominal Capacity = 20, 
treatments, 10 observations per 

treatment 



25.7 16.4 140 220 2043.1 7399.5 


a 

o2 

z 

Computed Power 

Simulated Power 

0.05 

53 

so. 42 

0.386 ± 0.0095 



certainly not allow all types of relevant combinations, In addition, 
this not Intended as ^ study of the economic practicality of doing 
analysis of variance for detecting change in capacity , only as an assess- 
ment of a method for approximating power. A study of how costly such 
anova procedures would be (in terms of number of replications) will be 
the subject of the next section. 

V. The Practical Use of Anova in Designed Simulation 
Experiments for Detecting Changes in Capacity 

All of the preceeding results were designed to lead us to this 
section. Here we discuss the economic implication of designing simu- 
lation experiments to detect differences in capacity from one condition 
to the other. The basic idea is to get some notion of how much simu- 
lation effort (number of touchdowns) is necessary in order to have 
a successful simulation experiment. Of course the key here Is to rely 
on the fact that we can conduct an analysis of variance and compute the 
power of the test on equality of capacity across conditions. We will 
asstune at the outset that a significance level of a - 0,05 would be used 
and that a successful test is one that would detect changes in capacity 
of 2-3 vehicles per hour with a high probability, say close to 0.95. 

In other words, we shall display how many touchdowns per con- 
dition are required in order that one achieve a power of the test of 
0.95 with a significance level of 0.05. It should be emphasized that 
this represents, by most standards, a very successful experiment. The 
procedure for computing the power Involves the use of theO.C. charts 
and the computation of $ as described in the previous section. In what 
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follows we attempt to tabulate requirements on number of replications 
that result in a successful or near-successful experiment. These tabu- 
lations will be followed by some general conclusions. Table IV gives 
results on n (number of observations per treatment) for various values 
of nominal capacity, number of treatments, that give specified power. 

Certain things become very obvious from the table. There is a more 
demanding sample size requirement for small capacities than for large 
capacities. The most demanding capacity is 30 (not 20) because 

z 

reaches its maximum at a value of approximately 30. In the case of 
comparing two treatments a fairly large number of replications is re- 
quired in order to achieve the high power when nominal capacity is 
below 60. It would seem reasonable that for the case of two treatments, 
a higher significance level should be used in the test. Even the case 
of 3 treatments can require a fairly costly number of replications for 
low capacities while for four and five treatments, the number of repli- 
cations required are not so demanding. 

The above would suggest that the most efficient simulation experi- 
ment might involve the use of more than one factor. For example, two 
factors with 3 levels each (equivalent to six treatments) would be com- 
parable to a study with 3 treatments and 3 n replications, where n is 
the number of observations in each combination. For experiments that 
must stay simple, say two or three treatments at low capacity, one must 
sacrifice something, perhaps using a somewhat higher significance level. 
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Table IV 


Sample Size Requirements for Specified Power 


Nominal 

Capacity 

No. of 
Treatments 

Significance 

Level 

Capacities 

n 

Power 

60 

5 

0.05 

56,58,60,62,64 

10 

0.83 

60 

5 

0.05 

56,58,60,62,64 

12 

0.93 

60 

5 

0.05 

56,58,60,62,64 

15 

>0.95 

60 

4 

0.05 

55,58,61,64 

10 

>0.95 

60 

3 

0.05 

57,60,63 

10 

0.62 

60 

3 

0.05 

57,60,63 

20 

0.93 

60 

2 

0.05 

58,62 

30 

0.84 

60 

2 

0.10 

58,62 

20 

0.82 

60 

2 

0.10 

58,62 

30 

0.94 

50 

2 

0.05 

48,52 

80 

>0.95 

50 

2 

0.10 

48,52 

70 

>0.95 

45 

5 

0.05 

36,38,40,42,44 

20 

>0.95 

45 

4 

0.05 

40,43,46,49 

25 

0.95 

45 

3 

0.05 

42,45,48 

60 

0.95 

30 

5 

0.05 

26,28,30,32,34 

30 

0.9 

30 

5 

0.05 

26,28,30,32,34 

35 

>0.95 

30 

4 

0.05 

25.5,28.5,31.5,34.5 

30 

0.93 

30 

4 

0.05 

25.5,28.5,31.5,34.5 

35 

>0.95 

30 

3 

0.05 

27,30,33 

60 

0.93 

20 

5 

0.05 

16,18,20,22,24 

20 

0.85 

20 

5 

0.05 

16,18,20,22,24 

25 

0.94 

20 

4 

0.05 

15.5,18.5,21.5,24.5 

20 

0.92 

20 

4 

0.05 

15.5,18.5,21.5,24.5 

25 

0.99 

20 

3 

0.05 

17,20,23 

50 

0.95 

20 

2 

0.05 

18,22 

70 

0.91 

20 

2 

0.05 

18,22 

80 

0.95 
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In this section we Illustrate the foregoing methodology with three 
examples that represent actual hypothetical simulation experiments that 
might be of Interest to NASA Langley personnel. (See Ref,. 2). 
example 1 

Suppose It Is of Interest to compare two controller conditions, 
condition A being M&S (metering and spacing) and condition B being 
vectoring. Suppose B has a capacity of 38.6 vehicles per hour, resulting 
from “ 93.3 seconds and = 18 seconds with Zg representing a 1% 

Gaussian Buffer. Condition A has a capacity of 46.7 vehicles per hour 
resulting from “ 77.1 seconds and = 11 seconds and Zg repre- 

sents a 1% Gaussian Buffer. Both A and B have s = 2 miles. In calcula- 
tion of the value of n, the number of arrivals required to give adequate 
power, we have 


a 


2 

Z 


C3600)2c2^^ 

»iAi 


51.803 


and thus 


4 


[(38. 6-42. 65)^+(46. 7-42. 65)^]n 
2(51.803) 


4 


(32.805)n 
103.6 


Using the O.C. Charts In Figure X gives n=»24 arrivals required per con- 
dition In order to achieve a power of 0.95 with a significance level of 
0.05. This is required In order that there be a good probability of 
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detecting differences between capacity at saturation when in fact the 
stated conditions A and B exist. These 24 arrivals will require approxi- 
mately 0 .6 hours per condition. 
example 2 

Consider two conditions A and B with both representing vectoring 
conditions with s = 2 miles for condition A and s = 3 for condition B. 

Allow Z„ to represent a 1% buffer for condition A and 5% for condition 
B, For condition A we have a capacity of 38.6 vehicles per hour with 
y “ 93.3 seconds, Otat ~ seconds, while conditions B has a capacity 
of 33,7 vehicles per hour, “ 106.75 seconds, and = 18 seconds. 

Using the same approach as Illustrated in example 1, we require n = 47 
in order to achieve a value of $ = 2,6 required to achieve a power of 
0.95 with a significance level of 0.05. This value of 47 arrivals requires 
approximately 1.3 hours per condition. 
example 3 

Let condition A represent metering and spacing with a capacity of 
46.7 vehicles per second, * 77.1 seconds, “ H seconds with a 

1% buffer. Let condition B represent Cockpit Display of Traffic Inforaa- 
tion (CDTI) , with a capacity of 52.9 vehicles per hour, “ 68 seconds, 

= 3 seconds, representing a near 0 % buffer, s = 2 miles for each 
condition, A value of n = 27 arrivals produces $ = 2.65, required for a 
power of 0.95. As a result, 0.6 hours would be required for each con- 
dition. 
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VII. Conclusions 


We have demonstrated that one can effectively use analysis of 
variance to do significance testing In simulation experiments where 
vehicle throughput Is the basic response. The difficulty with the fail- 
ure of the normality assumption presents no serious problem In conducting 
the statistical tests. In computing the power of the tests, one needs to 
make an adjustment in the computation of the non centrality parameter ♦ 

In using standard operating characteristic curves. The adjustment results 
in an approximate power and takes into account the heterogenous variances 
that are implicit In testing for differences In capacity between various 
conditions. 

For designing experiments that compare capacities, it turns out that 
in order to achieve an exceedingly high power for moderate changes In 
capacity (2-4 vehicles per hour), a moderate number of replications are 
needed when at least four or five "treatment combinations" are to be 
studied in the design. For only two treatments, the number of replica- 
tions are probably prohibitive, except for very high capacities. 
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